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Twenty-eight subjects were presented with computer generated 
grammatically deviant strings and asked to carry out two tasks on 
each of two experimental days* Task 1 was a forced-choice experi- 
ment in which 50 pairs of strings were presented aurally to each 
subject, and he had to select that member of the pair which he 
felt was the best approximation to a good English sentence. In 
Task 2, subjects were required to read and rank each string on a 
scale running from 1 (completely unacceptable) to 5 (completely 
acceptable). A different order of stimulus presentation was 
employed on each experimental dayj 14 subjects were assigned to 
one order on the first day and received the other order on the 



second day. 

Results show that subjects tend to prefer the same statement 
over orders and that rank and preference are highly correlated. 

There are considerable differences in preference among the 50 
pairs of stimulus items. Analysis of the data suggests that this 
task yields information relevant to the linguistic and in particular 
the ^syntactic competence of subjects when applied to grammatically 
deviant strings. Subjects appear to be trying to cope with the 
statements by comparing them to acceptable syntactic and/or semantic 

patterns . 
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STRUCTURAL APPERCEPTION IN THE ABSENCE OF SYNTACTIC CONSTRAINTS* 

Ronald S. Tikofsky 
University of Michigan 

Donald 6 , Reiff 
University of Rochester 

Psycholinguistics has devoted considerable attention to issues 
surrounding the concept of linguistic competence and linguistic per- 
formance, As part of the attack on the problem have come a series 
of papers which attempt to explain how a native speaker understands 
or processes the sentence, Osgood (1963) and Berlyne (1965) as 
as well as others hold to a chaining or mediatienal view to explain 
our ability to understand what we read or hear. The position 
taken by S-R psychology on this topic has been challenged by 
those closely aligned with the proponents of the new linguistics 
as proposed by Chomsky (1957) , later by Miller and Chomsky (1963)* 
Attempts have been made to test empirically hypotheses emanating 
from the transformational view of grammar. Many of these studies 
deal with the role of sentential complexity as a function of 
syntactic relations in the surface structure (Fodor and Garrett, 
1967; Maclay and Sleator, I960; Marks and Miller, 1964; Stoltz, 
*1967), These studies use fully grammatical utterances as stimuli 
and memory or a paraphrase as the response mode. Some attention 
(Downey and Hakes, 1968$ Miller and Isard, 1963) has been given 
to the speaker’s processing of putative sentence which are less 









2 



than fully grammatical# Tasks requiring the language user to make 
judgments of such strings may provide additional data on linguistic 
competence and its application# Such data can be obtained without 
requiring that the user be able to recall or paraphrase the stimulus* 

Saporta (1967) stafi ** chat, "a theory which purports t^ 
simulate the speaker^ ab* A y to distinguish between syntactically 
well-formed sentences e£ his language and the syntactically deviant 
ones must avoid making intuitively incorrect statements about what 
sentences are grammatical •” Recognizing that the living speaker- 
hearer is at best a poor approximation to the ideal speaker- 
listener of the theory, it seems reasonable to examine the language 
U 3 er f s processing of grammatically deviant strings# The assumption 
is that the language user will attempt to apply his linguistic 
knowledge to such strings in an effort to respond to them as 
though they were sentences of his language. 

It is generally agreed that normal discourse does not in fact 
consist of fully grammatical strings and complete sentences# Yet 
language users seem able to cope with less than fully grammatical 
and/or incomplete sentences with only little difficulty* Several 
strategies can be proposed to account for the apparent lack of 
interference in communicative efficiency resulting from such 
deviant utterances# The listener is of course in the position 
of being able to treat such utterances in terms of the total 

V 

context of the discourse, as well as the situational context, and 
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the utterances 1 overall structure* i,e. * semantic, syntactic and 
phonological features. However, attempts on the part of the 
language user to cope with single or paired grammatically deviant 
strings which are not part of discourse or normal communication 
situations would not be aided by either context. What could be 
said about strategies that might be employed by the language user 
in such eases? Some possible alternatives arc given below. 

If the user v ire presented with a single grammatically 
deviant string and he were to judge its approximation to an 
adequate grammatical sentence he might be expected to judge 
the utterance as being completely unacceptable* However, it 
is also possible that he might respond not to the totality of 
the utterance but rather to parts of it, or base his judgments 
on Ills perception and interpretation of the prosodic features. 
Other alternatives suggest that the user disregard some aspects 
of the utterance. He might impose an interpretation on the whole 
or on some portion analogizing to structural features which he 
considers nondeviant* In so doing, he would in effect be 
selectively attending to part of the utterance or some aspect 
of it, while relegating the remainder to the background. Thus, 
lie would treat the utterance as a u £igure-ground u problem in 
much the same manner as Gestalt psychology accounts for percep- 
tion. In this process, it is reasonable to assume that the 
figure will contain segments of the utterance which form either 
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a semantically or syntactically acceptable unit, or some combination 
o£ t ne two given a nondeviant phonological rendering. If this in 
fact were the way the language user operated on grammatically 
deviant utterances, then it would be predicted that those 
utterances containing both syntactic and semantic segments which 
can be interpreted as non-deviant would be ranked as close 
approximations to fully grammatical sentences. If either one 
or the other (but not both) aspects just noted were present, then 
the utterance would be ranked as somewhat less acceptable. 

Finally, utterances which could be treated as having only 
minimally adequate semantic or syntactic segments would be 
ranked as only minimally acceptable. Such considerations might 
well enter into the user's decision as to which member of a pair 
of strings would be selected as being the best approximation to 

an adequate graminatieal sentence. 

Ihe present study examines the judgments of native American 

speakers concerning the '-well-formedness" of random string of 
English sentenco-formatives presented as putative sentences. 
Speakers responded to those strings in two experimental conditions. 
In one, they were required to determine which member of a pair of 
'such strings was the better approximation to "good English," 
and in the other they were asked to rank each string in terms 
of its approximation to ."good English." Since the subjects were 
college students, it was assumed that they had achieved a 
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reasonably complete degree of linguistic knowledge. One of the 
major purposes of this research was to determine if subjects 
would show general agreement in their preference for a member of 
a pair of strings and how well they would agree on ranking each 

string. In addition, tae consistency of judgments over time was 

* 

studied. 



Method 

Stimuli. The stimuli were 50 pairs of gramr.ua tically deviant 
strings where the distribution of the forma tives was uniformly 
random. This precluded the recovery of any syntactic regularity. 
The 50 pairs were randomly selected from a set of 1000 pairs of 
computer generated strings of "equal syntactic length," each 
member of a pair had the same number of formatives. 

Pairs were generated from ten fully grammatical sequences in 
English having the form: 

determiner + adverb + adjective + noun + S + verb + S 
+ noun + preposition + noun# 

This yielded a 10 x 10 matrix, representing 100 grammatical 
English sequences. The computer program operated on the matrix 
in the following fashion: 

f 

1. it first selected on a random basis, a length between 
1 and 10$ 

2. it then selected randomly, items from the matrix} 

3. it continued to select items until a number of them 
representing twice the length had been reached; 
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4. it printed the selection of items with a space between 
the first and second halves of the string generated 

by step 3 above (this procedure yields pairs of state** 
ments of equal length) j and 

5. it continued this operation until the upper limit (1000 
pairs) had been reached. 

The 50 pairs (100 strings) used in this study are given i 
Table 1. In addition, this table also identifies the pair to 
which each string belongs in each of two orders, and the sequence 
in which they appear in the pair for the two orders. Sequence of 
string in a pair was random. Each order was presented to each 
subject in the manner described below. 

Each of the random orders was recorded on one channel of a 
two channel Ampex 351-2 tape recorder, with the number of the pair 
preceding each pair. Utterances were recorded with normal English 
sentence intonation. A signal was recorded on the second channel 
at the end of the second member of the pair. This signal stopped 
the tape playback. Stimuli were presented through an Ampex loud- 
speaker. These tapes were used in the forced choice preference 
task described below. 

4 Each string was also typed on a 3 x 5 card and bound in the 
order in which it appeared on the stimulus tape. These cards 
were used in the ranking task also described below. 
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Subjpctfl, # Twenty-eight University of Michigan students 
served as subjects* Each subject was randomly assigned to one 
of two groups, with fourteen in each group. One group of subjects 
responded to Order 1 on Day 1 and Order 2 on Day 2, while for the 
other group the orders were reversed on Days 1 and 2. There was 
a one day interval between experimental sessions. All subjects 
were individually tested. 

Test procedures* Forced ct oice preference task . Xn this 
phase of the experiment, pairs of strings were presented to the 
subjects as described above* Subjects were told that they would 
hear pairs of "statements that are not good English" and which 
probably would not make sense. They were told to select the one 
member of the pair which they felt to be the best approximation 
to "good English," by pushing one of two buttons on a response 
panel. The buttons were marked A and B to indicate the order of 
the statements in the pair, A being the first and B the second 
statement. Subjects took as long as they wished to respond. 

Choice was automatically recorded by a PDP-4 computer. When 
a response was completec, the playback was automatically reactivated 
and the next pair in the sequence was presented. 

* Test procedure i Ranking task . Upon completion of the pre- 
ference task, each subject ranked each of the 100 statements. 
Subjects were given the deck of cards on which each statement 
appeared and a response booklet containing the printed strings. 
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This booklet had five columns, numbered from one to five, to 
indicate degree of approximation to a "good English” sentence. 

The number 1 indicated complete unacceptability and 5 complete 
acceptability as a "good English” sentence. Subjects wnre told 
to rank each statement in terms of its approximation to a "good 
English sentence” by putting a check in the appropriate column. 

Results 

Preference data are given in Table 1. The columns headed 
"preference task” show the number of subjects who chose each 
string in a pair as the best approximation to a "good English 
sentence” for each order of presentation. Visual inspection 
of these data indicates that some statements are selected more 

frequently than others, while for some statements choices are 
not as well defined. If both statements in a pair were per- 
ceived as equally ungrammatical, then it would have been the 
case that the subjects would have selected either member of 
the pair equally often. This does not seem to be the case. For 
Order 1 there were 12 (24%) pairs where approximately 50% of the 
subjects chose each member of the pair, and for Order 2 this 
occurred for 20 (40%) pairs. Most strings (36) were preferred 
by 25-49% of the subjects, while only 15 preferred by 0-24% of 
the subjects and 19 were preferred by 75% or more of the subjects. 

The remaining 30 statements were preferred by 50-74% of the subjects. 
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Even this gross analysis of the preference data serves as a basis 
for concluding that the subjects were waking differential responses 
to the strings, and that there roust be something in the string 
which allows the subjects to show strong or weak preferences. 

Two other factors emerged from visual Inspection of these 
data. Subjects appear to prefer one button as against the other. 
However, there also appears to be a tendency for subjects to 
select the ‘tame string on both experimental days. These tendencies 
can be considered as competing, when a reversal in sequence in the 
pairs occurs over days, or as facilitory, when there is no change 

in position o'* the strings within the pair. 

When all preference data are combined there is a significant 
(X 2 * 10.0, p < .01) tendency to prefer the first member of a 
pair. For pairs not reversed across orders, there are 454 con- 
sistent (i.e., same member of the pair chosen on both days) as 
opposed to 209 inconsistent responses. When sequence within a 
pair is reversed, there are 476 consistent as opposed to 245 
inconsistent responses* This suggests that, on the whole, subjects 
tend to make their preference decisions in response to the strings 
ratiicr than by an arbitrary decision based on button preference or 
’position in the pair. 

Consistency of preference was also examined by determining 
the correlation of preference responses with orders and days. No 
obtained correlations were significant at or beyond the .01 level 
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of confidence. This Indicates that subjects exhibit similar pre- 
ference behavior over days, and further, the order of presentation 
of the pairs does not influence preference# 

Data for the ranking task are also summarized in Table 1, 
under the columns headed "ranking task," These columns give 
the percent subjects assigning a particular rank to a given 
string for each order. It must be remembered that for this task 
subjects saw the individual string one at a time, in contrast to 
listening to pairs of strings in the preference task. Inspection 
of the data reveals that there is no string which receives the 
same rank by all subjects, nor is there a string which does not 
have some proportion of the subjects assign less than three ranks 
to a statement. Of even greater interest is the fact that so 
many of the strings are assigned all five ranks. There is, 
nevertheless, a frequent preponderance of subjects who assign 
a particular rank to a string. Subjects, however, do not 
necessarily assign the middle rank to the string, but seem to 
be using the ranks as a device by which to interpret the accept- 
ability of the particular string. That the subjects are not 
always in agreement as to where a string lies with respect to 
*tlie criterion only suggests that they apply their linguistic 
knowledge in an idiosyncratic fashion to the task. Certainly 

these data do not reveal what it is that the subject responds to 

*** 

in making his judgment as to the grammatical adequacy of the 
string. 
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Individual subjects were remarkably consistent in their rank- 
ing. Correlations between ranks for days and order are significant 
at or beyond the .01 level. Further there is no evidence to 
indicate that the assignment of a rank to a string is a function 
of where it occurs (beginning or end) within the sequence of 
strings. An analysis of variance was carried out to determine xf 
there were day, list or order effects influencing the ranking 
behavior of the subjects. Rone of the effects tested by the 
analysis approached significance. 

Degree of correspondence between performance on the preference 
and ranking was also tested. The obtained overall correlations 
between rank and preference for the four experimental conditions 
are all statistically significant. Thus, when a subject showed a 
preference for a member of a pair, he tended to assign a high 
rank (4 or 5) to that string. When a member of a pair was not 
preferred it was usually assigned a low rank (1 or 2) . This 
result lends support to the hypothesis that subjects were using 
their ’’knowledge of English” in making preference and ranking 
decisions. 

<5 

, Discussion 

Statistical treatment of the data reveals that on the whole 
the subjects performed in a reliable and consistent manner in 
the experiment. It is also clear from the analysis that except 
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for a relatively few pairs, subjects tend to select one member of 
a pair as representing a closer approximation to "good English" 
than the other member* This is the case even when the procedures 
by which the stimuli were generated assured that each member of 
a pair have the same syntactic form, Further, the subjects can 
and do apply some differential judgments to each of the strings, 
even when given the opportunity to indicate that the strings are 
totally unacceptable, Of course, some strings do receive such 
judgments, but they do not represent the typical response to most 
of the statements. What the statistical analysis does not pro- 
vide is an explanation to account for the results, Rather, the 
analyses suggest that it is necessary to examine the data and 
stimuli in such a way so as to generate hypotheses which will 
provide such explanations, 

Several possibilities present themselves as bases for inter- 
pretation of the results reported. 

One approach assumes that a subject making the response 
closer to a "good English sentence" to a given string or 
putative sentence, contributes criteria in making the judgment 
that represent his ability to interpret and respond to strings 
of formativos in his, language. Such a notion is implied in 
Chomsky* a concept of "creativity" (1965) in terms of the language 
user's ability to handle novel sentences. The implications of 
this concept in relation to the issue of linguistic "competence" 



13 



is discussed by Koravcsik (1968)* Tills is also implied in the 
arguments developed by Saporta (1967)# Since* in the present 
study, all surface syntactic constraints were removed in the 
construction of the stimuli, the response required of the subject 
in terms of the judgment he makes may arise from one or both of 
two possible properties of the subjects 

a) one string in the pair possesses, for him, a greater 
degree of situational adequacy than the other (i*e*, 
it is semantically closer to "normal English") } or 

b) in the absence of a, the preferred string possesses 
a greater degree of grammatical adequacy than does 
the nonpreferred string (i.e., it is formally closer 
to normal English). 

Both a) and b) above carry the implication that the subject is 
responding to the entire string* This is not, as will be shown 
below, a necessary condition for the interpretation of the present 

findings* 

Assignment of properties such as those noted above to the 
listener represent competence considerations, which will retain 
the status of hypotheses* The act of interpretation on the part 
of the listener as he responds to the strings, although nonverbal, 
does Indeed focus a performance consideration with all the 
constraints which are typically imposed on performance. Thus, 
the data of the present experiment are performance data. This 
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leads to the conclusion that when a listener indicates his pre- 
ference for one string of a pair, he is in essence stating that 

either; 

a) he has previously responded to the string elsewhere, 
or to one similar to it, regardless of its degree or 
type of deviance; or 

b) if a) is not true, the preferred string might be 
considered more easily "adjustable" or interpretable, 
than the other member of the pair via projections 
made on it from the listener f s normal linguistic 
repertoire* 

The data do not of course reveal theoretically-expressible 
properties of the strings whose analogues are not within the 

individual subjects repertoire. 

A second approach by which the data may be interpreted 
also has to do with what are essentially performance-criteria. 

This approach assumes that the subjects expression of a pre- 
ference for a string is the result of his responding or attend- 
ing to some part or parts, but not all of the string. The 
possibility exists, that given a pair of nonsensical, agramraatical 
strings, the member of the pair possessing even the slightest 
suggestion of "normality" within the string will be preferred, 
since the subject is forced to choose one of the two strings. 
Consideration of the following pair illustrates his position; 
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5* "Meal sand soldier wine after the." 

6* "Show rice wine daughters delicious home." 

Categorial relations lor English determine that the only reasonable 
projection on the first statement for normality of element-ordering, 
is on the substring "after the." The remaining elements can be 
treated as a list. However, in the second statement there are 
many features of the structure of English, and perhaps, also 
features of a given subject’s experience, which might be projected 
upon the string. First, in terms of an individual’s experience, 
the string might appear to have "telegraphic" form, i.e#, deter- 
miners deleted for economy. On the other hand, it resembles, in 
its syntactic shape, either an imperative or an infinitive con- 
struct; something similar, say to; 

6a. "Show high society women beautiful jewelry" (in answer 
to a question, such as "What does George do for a 
living?" or, "Wha** are my duties?") in which case it 
resembles a truncated sentence; or perhaps it might 
be an imperative (in answer to the question, "What 
shall X do today?"). 

The relations between show— home . or show— daugh ters— home are 
perfectly normal sequences which can occur in a number of fully 
adequate sentences in English. The sequence rice wine is normal. 
Further, though grammatical relations are obscure, the collocation 
of wine— delicious is common. 
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Finally, in the first statement, the only possible colloca- 
tion of semantic items might involve meal— wine— af ter— the ; however, 
their ordering is far from normal, and this particular ordering 
would not appear in normal English even under the most complex 
of transformational relations. 

The above interpretation is not intended to explain the fact 
that the second string was preferred to the first by more than 
75 percent of the subjects. Rather, it simply serves to suggest 
that this string appears to accommodate more projections by the 
subject from a grammatical pattern that he doubtless knows. In 
addition, it exhibits a larger number of compatible semantic 
notions or situational responses and has more parts which, to the 
exclusion of the rest of the string, are either fully normal or 
closer to normal than anything found in the first string. Neither 
of these strings were assigned to one particular category on the 
ranking task by 50 percent or more of the respondents. However, 
there was a tendency to assign string 5 to positions which 
suggest that it is a poor approximation to "good English." On 
the other hand, there was a somewhat stronger tendency to assign 
ranks to string 6 indicating that it approximates "good English." 

f 

The ranking data can be interpreted in much the same way as 
the preference data. Strings such as the following! 

86. waiskeys buildingseses jobses daughters 
„58. food strong simplyseseseseses 

23. food those during woods thoses 
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are ranked by most respondents in the categories representing the 
poorest approximation to "good English*” However, strings such as 
the following? 

49. with outside show classes help 
4. young young buildings grow less hole milk 

33. man grows soldiers among several 
are assigned ranks indicative of close approximations to "good 
English," Strings 86, 58, and 23 offer little possibility for the 
subject to apply any of the criteria described in relation to the 
preference data in such a way as to allow the strings to be con- 
sidered as good approximations to English, This is not the case 
for strings 49, 4, and 33. These strings and others ranked as 
being good approximations to English can be easily segmented so 
that some sequence of elements within the string meet or nearly 
meet the criteria noted above. 

It is also interesting to note that even in those cases 
where one member of a pair was highly preferred, the two strings 
may be assigned the same rank. However, in other instances, the 
preferred member was ranked as being an adequate approximation to 
"good English" but the other member of the pair a poor approxima- 
tion. Both members of the first pair given below were called 
good approximations, but the second item was preferred 3 to 1. 
Exactly the same situation exists for the second pair below, except 

that both members were ranked as poor approximations to good English. 

« 
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Xn the third pair, both ranking and preference were divided. Here 
the preferred member of the pair was ranked as a close approxima- 
tion to "good English' 1 at a 10 to 1 ratio over the member ranked 
as a poor approximation. The sample pairs are; 

31. food show 

X 

32. above wine 

23. food those during woods thoses 

2 

24. old rice good delicious grow strong eat 

3 

93. group seven simply green 

4 

94. above bigs meal 

Even on the basis of the present analysis of the data it 
seems reasonable to argue that native speakers can and do inter- 
pret random strings of forma tives in their language. Further, 
such interpretations are made even when it is known that there 
are no constraints whatsoever within the strings themselves. 

Thus, even when the "sender" is encoding no message, the "receiver" 
does not receive a non-message. How this reconstruction is per- 
formed and on what levels, remains obscure, but it is reasonable 
to hypothesize that syntax, lexicon, and meaning are to some 
degree involved. 

The results of the present study lend support to the argument 
that an organism having acquired a grammar can at some point in 
its history respond to any string in the language in terms of the 
internalized grammar. That the subjects had no experimentally 

f 
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induced criteria of ’’acceptability" or ’’good English” strengthens 
this claim* Thus, what has been demonstrated in the present experi~ 
went is that the subjects do in fact make reliable and relatively 
consistent judgments in response to putative linguistic material 
which contributes little to these judgments. Considering the 
subjects 1 decisions as dependent variables, it seems clear from 
the data presented in this paper that there are Indeed independent 
variables which influence them. However, the nature, function and 
Interactions of these variables still need to be identified and 
subjected to further study. 
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Education, Contract OEC-3-6-Q61784-Q508, and ONR Contract NQ0014- 
68A-0Q91(CNR) . A preliminary version of the paper was presented 
at the summer 1967 meetings of the Linguistic Society of America. 
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1 

Preferred string-bo th members ranked as good approximation. 

o 

Preferred string-both members ranked as poor approximation. 

3 

Preferred string-ranked as good approximation, 

^Non-preferred string ranked as poor approximation. 
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PAIR SO. 


STRING 


°1 


°2 




1A 


38B 


1 


IB 


38A 


2 


2A 


22k 


3 


2B 


225 


4 


3A 


44A 


5 


3B 


44B 


6 


4A 


11B 


7 


4B 


11A 


8 


5A 


36B 


9 


5B 


36A 


10 


6A 


4B 


11 


6B 


4A 


12 


7A 


19 A 


13 


7B 


19B 


14 


8A 


21B 


15 


8B 


21A 


16 


9A 


45B 


17 


9B 


45A 


18 


10A 


12B 


19 


10B 


12A 


20 


Ilk 


26A 


21 


11B 


26B 


22 


12A 


6B 


23 


12B 


6k 


24 


13A 


3B 


25 


13B 


3A 


26 



TABLE 1 

Grammatically deviant strings, orders of presentation, preference ai 
mo., STRING 



Fcws make home book old professors man. 

, With drink the young knowledge primarily especially amcngs. 

& 

, Whiskeys straws wine makes daughters, 
f Young young buildings grow less hole milk. 

Meal sand soldier wine after the. 

Show rice wine daughters delicious home. 

New show these extremely. 

Rices above a. 

Intelligence news good deliciouses many strong. 

Extremely primarily beautiful foods food windows lakes. 

The milk those rice. 

Around milk paint around. 

Around drink machines windows more book the jobs. 

Many especiallys big ugly machineseses. 

Help quiteses extremely windows paint lady sand. 

During sand paint essentials good man above big. 

Ver>s. 

Make among. 

Group deliciouses paint some outside. 

Lady sand through grow soldiers. 

Home dishes. 

Eats. 

Food those during woods thoses. 

Old rice good delicious grow strong eat. 

With professors lakes drink. 

Seven seven man seven machines view. 



o 

ERIC 



TABLE 1 



at Ion, preference and ranking score*? for Order 1 and Order 2 



r especially among s. 
Lk. 



strong. 

d windows lakes. 



Uie jobs. 

ady sand, 
beve big. 



PREFERENCE DATA 1 



15 


20 


13 


8 


9 


5 


19 


23 


9 


3 


19 


25 


22 


22 


6 


6 


9 


12 


19 


16 


8 


6 


20 


22 


22 


26 


6 


2 


15 


14 


13 


13 


12 


9 


15 


19 


11 


13 


17 


15 


18 


16 


10 


12 


6 


8 


22 


19 


17 


6 


11 


18 



ranking 
12 3 4 5 



04 21 60 14 — 
07 21 43 28 — 

07 14 28 43 07 

— 07 10 40 43 

18 40 18 18 07 

— 18 28 25 28 

10 14 36 28 07 
10 14 32 40 04 

32 39 24 — 04 
36 39 i4 07 04 

10 57 21 10 — 

— 10 36 32 36 

04 25 42 24 04 
40 36 18 07 — 

32 32 10 24 ~ 
10 57 14 U 04 

10 36 28 41 04 

— 07 18 64 10 

07 24 25 32 10 
04 21 32 28 07 

04 04 18 24 50 
07 18 10 10 54 

24 60 07 07 — 
10 54 14 14 07 

— 24 24 39 14 
04 24 18 46 07 




DATA 

1 2 3 4 5 



10 18 32 28 10 
10 27 21 25 10 

04 07 60 24 10 

— 14 32 36 21 

10 28 40 21 — 
18 14 40 14 14 

— 32 21 40 07 
14 32 18 21 14 

18 39 14 28 — 
32 40 18 10 — 

14 28 36 14 07 
04 39 21 36 — 

07 10 36 28 18 
36 43 18 04 — 

24 50 10 14 — 
22 24 39 07 07 

18 36 10 21 14 
18 32 21 10 18 

07 21 36 25 10 
07 32 36 18 07 

07 24 04 24 39 
18 14 14 14 40 

25 50 18 04 04 
04 43 36 18 — 

04 04 18 64 14 
— 28 42 28 — 



Table 1 (cont.) 



BAIR SO. 


STRING NO 


°1 °2 




14A 37B 


27 


14B 37A 


28 


15A 7B 


29 


1515 7A 


30 


16A 14B 


31 


16B 14A 


32 


17A 10A 


33 


173 10B 


34 


ISA 32B 


35 


18B 32A 


36 


19A 43A 


37 


19B 43B 


38 


20A 8A 


39 


203 8B 


40 


21A 5A 


41 


21B 5B 


42 


22A 16B 


43 


22B 16A 


44 


23A 23A 


45 


23B 23B 


46 


24A 20B 


47 


24B 20A 


48 


25A 31B 


49 


25B 31A 


50 


26A 34A 


51 


26B 34B 


52 



STRING 



Make delicious eats. 

Through prefer few delicious. 

Professors group group. 

Quite dishes meal after. 

Food show. 

Above wine. 

Man grows soldiers among several. 

Help view daughters ins whiskey dishes. 

Home make rices showsesefc intelligence. 

Quite jobs strong machines very drinks straw help. 

Very with make extremely the windowses. 

Newses very delicious man. 

Grow makes rice. 

Building quite soldiers. 

Especially old. 

Lakeses. 

Rices greens grow view. 

Around afe^ve lady like fields lady. 

Book bigs during green prefer jobses. 

Prefers daughters groups the drinks. 



Woods . 

Machines classes. 

With outside show classes help. 

Classes machines building intelligence especially. 



Soldier group raakeses wood man dishcses. 

Simply paint make with these daughters news group. 



# 

* 



)’at>Xe 1 (cent,) 



tee, 

3 straw help. 

«£? # 



nee especially, 
icocs . 

ers news group. 



PREFERENCE DATA 



RANKING DATA 

1 2 3 4 5 1 2 3 4 5 



22 


20 


07 07 14 36 36 


04 28 39 41 07 


n 

1 


6 


8 


10 36 32 21 — 


— 28 46 14 10 


1 

if 

f 


20 


19 


10 36 24 28 


04 18 39 28 14 


l 

K 

it 


8 


9 


04 43 43 04 07 


07 54 28 10 — 


n 

# 


9 


7 


— 07 21 32 39 


04 28 14 21 32 


J 


19 


21 


07 18 40 36 


07 24 07 28 32 


| 

* 


24 


25 


— 14 18 28 39 


— 07 32 40 21 


8 

% 


4 


3 


10 24 32 41 10 


07 40 40 14 — 


# 


13 


14 


28 36 28 07 — 


07 28 41 36 07 


A 


15 


14 


14 54 28 04 — 


14 46 24 07 04 


1 


10 


14 


14 54 18 14 — 


10 46 28 14 — 




18 


14 


18 32 28 18 04 


18 36 40 — 07 


# 


22 


23 


— 14 10 36 40 


«■»*" 18 43 32 07 


u 


6 


5 


07 21 36 18 18 


04 36 36 21 04 


1 


25 


26 


04 18 18 21 40 


10 21 10 21 36 


W 

!'! 


3 


2 


36 28 28 07 — 


39 24 25 — 10 




15 


19 


— 21 32 40 07 


— 28 39 21 10 


1 


13 


9 


14 28 24 28 04 


07 32 40 14 07 


it 


g 


9 


— 40 28 24 07 


07 24 32 21 14 




22 


19 


07 28 32 25 07 


04 39 36 21 — 


¥ 


18 


18 


04 10 10 21 54 


18 24 14 07 36 


ft 

% 


9 


10 


07 14 21 46 10 


07 18 40 28 07 




20 


27 


— 04 18 57 21 


"" 04 21 43 32 


1 

< 


8 


1 


07 32 42 18 — 


04 21 18 57 — 


y 

« 

t 


4 


8 


28 32 28 10 — 


14 32 36 18 — 


i 


24 


20 


07 40 32 21 — 


04 43 28 22 04 


i 



h 

i* 

I 



o 

ERIC 



Table 1 (cont.) 



PAX R 
°1 


NO. 

°2 


STRING 


27A 


2A 


53 


27B 


2B 


54 


28A 


46A 


55 


28B 


46B 


56 


29A 


30B 


57 


29B 


30A 


58 


30A 


41A 


59 


30B 


41B 


60 


31A 


40A 


61 


313 


40B 


62 


32A 


9A 


63 


32B 


9B 


64 


33A 


Ik 


65 


333 


IB 


66 


34A 


13A 


67 


34B 


13B 


68 


35 A 


24A 


69 


35B 


24B 


70 


36A 


48B 


71 


36B 


43A 


72 


37A 


35A 


73 


37B 


35B 


74 


38A 


25B 


75 


383 


25A 


76 


39A 


17A 


77 


39B 


17B 


78 



STRING 



Home steel classes. 

Beautifuls extremely. 

This book daughters. 

Lakes new totally. 

Someses like lakes few windows ugly. 

Food strong simplyseseseseses. 

Strong very through grows a jobs very. 

Outside very less sand help less strong ugly. 

The beyond knowledge thoses building seven intelligence. 
Windowses holes arounds help primarily. 

Paint esp # ciallys . 

Essentials dishes jobs. 

After totally. 

Buildings . 

Windows intelligences strong soldier seven hole intelligence. 
After some especially help lady daughters new sand. 

Big home. 

Those several. 

Beyond primarily seven fields this. 

Prefer intelligence grow big make. 

View group intelligences. 

Beautifuls young man. 

Especially a sand helps these extremely view the. 

Leases cat totally thoses with beautiful these. 



Prefer jobs less man. 

Green primarily especially wood. 



Table 1 (cent.) 








PREFERENCE DATA 


RANKING 




°1 


°2 


1 2 3 4 5 




24 


25 


— 18 21 50 10 




4 


3 


36 28 25 07 04 




11 


14 


04 21 28 28 18 




17 


14 


— IS 47 28 07 




27 


27 


21 36 25 14 04 




1 


1 


54 39 07 — — 


y* 

ong ugly. 


17 


14 


18 40 04 36 04 


11 


14 


10 54 24 07 04 


; seven intelligence, 
.ly . 


16 

12 


15 

13 


14 40 24 18 04 
10 36 39 10 04 




11 


14 


10 28 24 28 07 




17 


14 


04 43 24 21 07 




12 


14 


— 18 32 32 18 




16 


13 


— 14 14 14 57 


r seven hole intelligence, 
ghters new sand. 


6 

12 


8 

20 


18 28 24 28 — 
04 36 28 28 04 




21 


25 


— 10 18 21 50 




5 


3 


04 14 40 32 10 




11 


14 


— 32 14 43 10 




17 


14 


— 28 36 28 07 




19 


23 


— 10 14 43 32 




9 


5 


07 32 60 14 — 


Iiaoly view the. 
itiful these. 


11 

17 


20 

8 


10 32 32 25 — 
24 28 18 24 04 




11 


16 


— 18 28 46 07 




11 


12 


— 24 36 28 10 



DATA 

12 3 4 5 



— 21 25 36 28 
21 47 25 07 — 

07 04 28 43 18 
07 14 28 47 07 

21 32 24 18 04 
68 24 07 

10 57 18 10 04 
18 18 57 07 — 

10 32 28 25 04 
07 28 40 14 10 

10 36 39 10 04 
14 24 40 18 — 

10 36 14 28 10 
14 22 — 18 46 

18 42 22 14 04 
14 21 28 28 07 

07 24 18 04 46 
07 26 32 21 14 

— 28 28 36 07 

— 50 28 21 — 

— 07 36 28 28 
14 36 21 22 04 

04 18 46 24 07 
25 36 14 24 — 

04 28 43 18 07 
04 36 28 39 10 



/ 



Table 1 (cent.) 



FAIR 


NO. 


STRING NO. 


STRING 


°X 


°2 






40A 


ISA 


79 


Several rice during book outside hole. 


4CB 


15B 


80 


Quite thrcughsbs prefer daughters. 


41A 


50A 


81 


Help outside whiskey professor good extremelys with. 


41B 


503 


82 


More amongs classes windows es above big. 


42A 


39A 


83 


Steel fields knowledge eat wood. 


423 


39B 


84 


In shows seven around. 


43A 


42B 


85 


Prefer building daughters meal classes few withs meal. 


43B 


42A 


86 


Whiskeys buildingseses jobses daughters. 


44A 


18B 


87 


Classes beyond with green. 


443 


18A 


88 


Machines seven the home. 


45A 


33B 


89 


Uglys prefer good a these deliciouseses. 


45B 


33A 


90 


Few beyond fields simply bo'ik after big hole. 


46A 


49B 


91 


After help machines disheses mores buildings. 


46B 


49A 


92 


Aboves soldier several group professor foods ugly. 


47A 


29A 


93 


Group seven simply green. 


47B 


29B 


94 


Above big3 meal. 


48A 


27A 


95 


Show after several help professors above. 


48B 


27B 


96 


Seven preferseses good view. 


49A 


28B 


97 


Verys. 


49B 


28A 


98 


With wine. 


50A 


47B 


99 


Several theses prefer eat group simply 


SOB 


47A 


100 


Strong the meal drink during quite classes 



0, - Order 1 



0 2 - Order 2 



^Numbe'r of subjects selecting the string over days for each order. 

^Percent subjects assigning a given rank for each string over days for each orde 



Table 1 (eont.) 



IrP # 



oKtresaelys with. 

big. 



»ges few withs meal, 
iters. 



ie«es. 
r big hole. 

buildings. 

>ss©r foods ugly. 



i above. 



imply 
e classes 



ih order. 




PBEF2REMCE DATA 


ranking 


°x 


°2 


1 2 3 4 5 


12 


13 


04 36 40 18 04 


16 


15 


14 14 28 39 04 


19 


15 


21 36 32 10 — 


9 


13 


10 50 24 14 — 


11 


7 


04 07 32 46 10 


17 


21 


07 10 28 43 10 


23 


16 


14 54 21 10 — 


5 


10 
Jk ** 


46 46 04 04 — 


12 


15 


39 39 21 


16 


13 


— 07 24 43 24 


5 


9 


2? 60 07 07 04 


23 


19 


— 10 36 39 14 


14 


13 


32 24 24 18 — 


14 


15 


14 46 24 14 — 


26 


25 


— 10 21 54 14 


2 


3 


04 43 39 14 — 


17 


16 


— 18 46 36 — 


11 


12 


18 28 32 18 04 


6 


5 


10 43 25 14 07 


22 


23 


18 25 57 


11 


16 


04 36 32 21 04 


17 


12 


10 46 28 14 — 



DATA 

1 2 3 4 5 



10 25 18 32 14 
14 32 14 32 07 

21 54 21 04 — 
14 46 36 04 — 

04 18 28 53 10 
04 28 42 14 10 

07 50 32 10 — 
50 24 14 10 — 

04 18 25 42 10 
04 18 28 28 21 

07 36 43 14 — 
04 07 18 57 14 

10 36 39 10 04 
21 43 18 18 — 

— 21 32 28 18 
18 32 28 21 — 

07 32 32 28 — 
10 21 36 24 07 

32 46 10 10 — 
07 24 10 24 32 

18 36 32 14 — 
10 24 50 14 — 
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